A Probabilistic Framework for Resource-Constrained Multi-Agent Planning

In this paper, we consider the problem of temporally coordinating the resource demands of a set of independent agents. We assume that resources are unreliable, making it necessary to retain imprecision in the execution times assigned to specific agent operations. To this end, a probabilistic model of resource allocation is developed for use in estimating the consequences of execution intervals (representing sets of possible resource allocation decisions). This leads to a probabilistic representation of requests for resource usage for which resource congestion constraints can be defined. We consider two applications of the framework: prediction of bottleneck resources and time bound scheduling. I COORDINATING RESOURCE USAGE IN MULTI-AGENT PLANNING Multi-agent planning involves coordination of the operations of a set of independent agents so as to achieve the goals of each individual agent. Of central importance here is consideration of the possible interactions among agents, which can lead to situations of mutual advantage (in the case of cooperative agents) or interference (if agents have conflicting objectives and do not cooperate). Research in multi-agent planning [4, 8] typically assumes that the goals being pursued by agents relate strictly to the achievement of a certain state. Emphasis is placed on determining "how" this state can be achieved in the presence of other agents. In time-constrained problem domains, however, the goals of agents often relate more to "when" a specific state can be achieved than "how". The problem of factory scheduling, which has been the context of our work, is perhaps an extreme example of this situation. Parts must be produced for multiple orders (agents) to meet imposed deadlines, minimize production time and satisfy other factory objectives, and the primary source of interactions among agents is the need to share resources (e.g. machines, operators, tools). Interactions among agents in this domain very rarely prevent achievement of each agent's desired state (i.e. the finished parts) but can significantly affect the circumstances under which this desired state is achieved (e.g. the finish date, the duration of the production process, etc J. Dealing with these types of interactions is the subject of this paper. Assuming a cooperative planning framework, there is much to be gained by anticipating situations of resource contention and attempting to temporally coordinate requests for these resources. Analyses of projected resource demands can guide individual agents in planning to achieve their goals (e.g. specific resources to avoid when possible). Advance coordination of demands for heavily utilized resources can maximize the common utility of agent's plans (e.g. make the most efficient use of shared resources) [10]. An important *This research was supported in part by the Air Force Office of Scientific Research under contract F49620-82-K-0017 and the Robotics Institute, and was performed while Nicola Muscettola was a visiting scholar in the Intelligent Systems Laboratory of the Robotics Institute. Stephen F. Smith The Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 USA issue, of course, is the level of detail at which "deals" among agents concerning future resource usage should be made. While we might expect that precise temporal coordination of requests for each shared resource would lead to the best possible plans, such plans ignore the dynamics of the planning environment. Resources are characteristically unreliable (e.g. susceptible to unanticipated periods of unavailability, sometimes fail to produce the desired effect, etc.), and consequently, advance commitment to a detailed course of action and timetable is of little use. Agents should seek less precise agreements concerning resource access that can usefully guide plan refinement as the external environment allows and requires it. Less detailed agreements among agents require more abstract problem descriptions. To this end, we can formulate problems in which agent operations require capacity on aggregate resources (representing functional groupings of individual resources) over their expected durations, and we wil l assume in this paper that both the amount of capacity required by a given operation and its duration are fixed approximations of more detailed suboperation characteristics. We also assume that resource capacity is required throughout an operation's duration. As with the identity of the resource that wi l l be used to perform a given operation, we would like to remain imprecise with respect to operation execution times. This, however, presents a difficult problem. Classical deterministic scheduling methods [1] must make specific allocation decisions in order to take resource capacity constraints into account This defeats much of the purpose of abstracting in the first place since the resulting plans wil l designate a single point in the temporal dimension of the abstract solution space. We might consider generalizing these temporal constraints to delineate sets of possible allocation decisions after the fact. However, given the high degree of interaction between allocation decisions, it is difficult to imagine how this could be accomplished in any meaningful fashion. We might also consider reasoning with a coarser granularity of time at the aggregate level (e.g. time steps of hours instead of minutes), but this also introduces temporal imprecision in a fairly arbitrary fashion. In this paper, we present an approach to reasoning with temporally imprecise requests for resource capacity (representing sets of possible allocation decisions) that provides a oasis for producing more meaningful aggregate plans. This is accomplished by adopting a probabilistic view of resource allocation and injecting randomness into the decision-making process. At the same time, we can describe characteristics of the stochastic allocation process that enable the definition of consistency constraints analogous to those that would result from a deterministic model. Furthermore, we can bias the stochastic process to reflect the strategies and preferences of the actual deterministic allocator (i.e. the generator of final decisions for actual execution). II PROBABILISTIC RESOURCE ALLOCATION As stated above, we propose the use of a probabilistic model of resource allocation as a means of reasoning about sets of possible allocation decisions. We are interested in a Muscettola and Smith 1063 mechanism for evaluating an abstract, temporally underconstrained set of plans with respect to its expected requests for resource capacity. Our approach is to develop a random model of the actual resource allocation process, ana to use the probabilistic characteristics of randomly generated allocation decisions as the basis for plan evaluation. In modeling resource allocation as a stochastic process, it is crucial tnat the random allocator take into account not only the restrictions specified in the abstract plan, but also the preferences and strategies of the actual deterministic allocator. Thus, we wi l l conceptualize the allocation process as consisting of a random start time generator (RSTG) that stochastically selects a specific allocation for each operation in accordance with this larger set of constraints, which we wil l call the RSTG-constraints. We wi l l refer to the set of allocation decisions produced by RSTG as the RSTGallocations. A. Constraining the Random Allocation Mechanism We distinguish three different types of RSTG-constraints: 1. temporal constraints These constraints refer to the earliest start time and latest end time restrictions associated with each operation opr Since we assume that each opi has a fixed duration, designated henceforth as bopp these constraints define the set of the possible start times for each opp STI(opt) (Start Times Interval), an interval of time bounded by est(op), the operation's earliest start time, and lst(op), its latest start t ime** The temporal constraints, and consequently the ST/s, associated with operations belonging to the sameplan must be mutually consistent. Consider Figure 1, where STl(op1) is depicted in the context of a two step plan. In this case, est(op2) must not precede r, in time, since STl(op2) would otherwise contain allocation possibilities that would introduce a conflict Similarly, Ist(op)2 must not precede t2 in time. These consistency conditions are precisely those that are maintained in many existing temporal planners (e.g. [11]). STI(op) must also be consistent with currently imposed resource capacity limitations. Such consistency conditions are discussed in Section IV. 2. allocation strategy This constraint dictates the sequence in which allocation decisions wil l be generated. It specifies a partial ordering of all the operations constituting the planning problem. For example, a plan by plan in priority order strategy dictates that all operations of a given agent's plan wil l be considered before those of any agents with lower priority. Operations belonging to the same plan are ordered according to a given plan scheduling strategy (e.g. forward from the first operation). 3. preference constraints These constraints define To simplify the presentation below, we assume STl(op) to be a connected interval. The framework can be easily generalized to accommodate a discontinuous set of possible operation start times. Figure 2: A two operation plan with op1 starting at time t1 objectives and concerns that influence actual allocation decisions. As has been pointed out in previous research [2, 3], we can model specific allocation preferences as real functions over time that estimate the relative desirability of various allocation decisions. For example, if an operation has a fixed due date with a delay penalty, we can express the tardiness constraint as a function that decreases after the due date at a rate proportional to the marginal tardiness loss. Constraints on finishing early (e.g. factory holding costs) can be expressed similarly, with the utility function increasing until the due date is reached in this case. The formulation of other allocation preferences depends not only on problem characteristics (as do the above examples) but also on prior allocation decisions. A desire to minimize idle time between operations, for example, can be expressed with a utility function that decreases from the current earliest start time of a given operation onward. However, this constraint is meaningful only in the case of an op. that is preceded in the allocation strategy by one or more upstream operations in op's plan, and its exact structure can be defined only when actual allocation times have been determined for these preceding operations. Nonetheless, all active constraints are known when making a specific allocation decision, and we can therefore obtain their combined "level of satisfaction". B. Modeling the Behavior of the Random Allocation Process We can derive, for each op the probability of it starting around a certain instant of time in STI(op), expressed by the density Pft(oprt). This is of central importance to our approach, as we wil l see in Section I I I when we consider the representation of a resource's available capacity. Here, we consider our model of the behavior of RSTG and the derivation of As allocation decisions are generated by RSTG, it is quite possible that STI(opt) for a given opi wil l become further constrained. In Figure 2, for example, the decision to schedule op1 at r1 constrains RGST to start op2 after t We wi l l refer to actual set of possible allocation times for op. at the point it is considered by RSTG as ASTl(pp) (Actual Start Times Interval). Since ASTI(op) is a function of random decisions, its limits are themselves random variables. RSTG wil l select a start time in ASTI(op) according to a choice rule, a density of probability that defines, for every point in ASTI(op)% the likelihood of the start time falling in its neighborhood. The choice rule is constructed to reflect the combined value of all active preference constraints according to the following criterion: Choice Rule Criterion: Given two possible start times t1 and t2 if the value of the combined constraints at t1is c times that in t2 the likelihood of selecting t1 will be c times that of t2 If ASTl(op) is the interval we indicate the choice rule for opi with If v(r) is the value in t of the combined active constraints, the Choice Rule Criterion becomes